Face Generation¶

In this project, you'll use generative adversarial networks to generate new images of faces.

Get the Data¶

You'll be using two datasets in this project:

MNIST
CelebA

Since the celebA dataset is complex and you're doing GANs in a project for the first time, we want you to test your neural network on MNIST before CelebA. Running the GANs on MNIST will allow you to see how well your model trains sooner.

If you're using FloydHub, set data_dir to "/input" and use the FloydHub data ID "R5KrjnANiKVhLWAkpXhNBe".

data_dir = './data'

# FloydHub - Use with data ID "R5KrjnANiKVhLWAkpXhNBe"
#data_dir = '/input'


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import helper

helper.download_extract('mnist', data_dir)
helper.download_extract('celeba', data_dir)

Downloading mnist: 9.92MB [00:01, 9.00MB/s]                            
Extracting mnist: 100%|██████████| 60.0k/60.0k [00:05<00:00, 11.6kFile/s]
Downloading celeba: 1.44GB [08:14, 2.92MB/s]

Extracting celeba...

Explore the Data¶

MNIST¶

As you're aware, the MNIST dataset contains images of handwritten digits. You can view the first number of examples by changing show_n_images.

show_n_images = 25

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
%matplotlib inline
import os
from glob import glob
from matplotlib import pyplot

mnist_images = helper.get_batch(glob(os.path.join(data_dir, 'mnist/*.jpg'))[:show_n_images], 28, 28, 'L')
pyplot.imshow(helper.images_square_grid(mnist_images, 'L'), cmap='gray')

<matplotlib.image.AxesImage at 0x7ff5ab20ddd8>

CelebA¶

The CelebFaces Attributes Dataset (CelebA) dataset contains over 200,000 celebrity images with annotations. Since you're going to be generating faces, you won't need the annotations. You can view the first number of examples by changing show_n_images.

show_n_images = 25

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
mnist_images = helper.get_batch(glob(os.path.join(data_dir, 'img_align_celeba/*.jpg'))[:show_n_images], 28, 28, 'RGB')
pyplot.imshow(helper.images_square_grid(mnist_images, 'RGB'))

<matplotlib.image.AxesImage at 0x7ff5ab4e6400>

Preprocess the Data¶

Since the project's main focus is on building the GANs, we'll preprocess the data for you. The values of the MNIST and CelebA dataset will be in the range of -0.5 to 0.5 of 28x28 dimensional images. The CelebA images will be cropped to remove parts of the image that don't include a face, then resized down to 28x28.

The MNIST images are black and white images with a single color channel while the CelebA images have 3 color channels (RGB color channel).

Build the Neural Network¶

You'll build the components necessary to build a GANs by implementing the following functions below:

model_inputs
discriminator
generator
model_loss
model_opt
train

Check the Version of TensorFlow and Access to GPU¶

This will check to make sure you have the correct version of TensorFlow and access to a GPU

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
from distutils.version import LooseVersion
import warnings
import tensorflow as tf

# Check TensorFlow Version
assert LooseVersion(tf.__version__) >= LooseVersion('1.0'), 'Please use TensorFlow version 1.0 or newer.  You are using {}'.format(tf.__version__)
print('TensorFlow Version: {}'.format(tf.__version__))

# Check for a GPU
if not tf.test.gpu_device_name():
    warnings.warn('No GPU found. Please use a GPU to train your neural network.')
else:
    print('Default GPU Device: {}'.format(tf.test.gpu_device_name()))

TensorFlow Version: 1.8.0
Default GPU Device: /device:GPU:0

Input¶

Implement the model_inputs function to create TF Placeholders for the Neural Network. It should create the following placeholders:

Real input images placeholder with rank 4 using image_width, image_height, and image_channels.
Z input placeholder with rank 2 using z_dim.
Learning rate placeholder with rank 0.

Return the placeholders in the following the tuple (tensor of real input images, tensor of z data)

import problem_unittests as tests

def model_inputs(image_width, image_height, image_channels, z_dim):
    """
    Create the model inputs
    :param image_width: The input image width
    :param image_height: The input image height
    :param image_channels: The number of image channels
    :param z_dim: The dimension of Z
    :return: Tuple of (tensor of real input images, tensor of z data, learning rate)
    """
    # TODO: Implement Functio
    real_input = tf.placeholder(tf.float32, shape=[None, image_height, image_height, image_channels], name='real_input')
    z_data = tf.placeholder(tf.float32, shape=[None, z_dim])
    learning_rate = tf.placeholder(tf.float32)

    return real_input, z_data, learning_rate


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_inputs(model_inputs)

Tests Passed

Discriminator¶

Implement discriminator to create a discriminator neural network that discriminates on images. This function should be able to reuse the variables in the neural network. Use tf.variable_scope with a scope name of "discriminator" to allow the variables to be reused. The function should return a tuple of (tensor output of the discriminator, tensor logits of the discriminator).

def discriminator(images, reuse=False):
    """
    Create the discriminator network
    :param images: Tensor of input image(s)
    :param reuse: Boolean if the weights should be reused
    :return: Tuple of (tensor output of the discriminator, tensor logits of the discriminator)
    """
    # TODO: Implement Function
    with tf.variable_scope('discriminator', reuse=reuse):
        
        # Input is an image: 28x28x3(or 1)
        c1 = tf.layers.conv2d(images, 64, [5,5], [2,2], activation=None, padding='same')
        c1 = tf.maximum(0.2*c1, c1)
        
        # Input is 14x14x64
        c2 = tf.layers.conv2d(c1, 128, [5,5], [2,2], activation=None, padding='same')
        c2 = tf.layers.batch_normalization(c2, training=True)
        c2 = tf.maximum(0.2*c2, c2)
        
        # Input is 7x7x128
        c3 = tf.layers.conv2d(c2, 256, [5,5], [2,2], activation=None, padding='same')
        c3 = tf.layers.batch_normalization(c3, training=True)
        c3 = tf.maximum(0.2*c3, c3)
        
        # Input is 4x4x256
        c3_flat = tf.reshape(c3, [-1, 4*4*256])
        logits = tf.layers.dense(c3_flat, 1)
        output = tf.sigmoid(logits)
        
    return output, logits


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_discriminator(discriminator, tf)

Tests Passed

Generator¶

Implement generator to generate an image using z. This function should be able to reuse the variables in the neural network. Use tf.variable_scope with a scope name of "generator" to allow the variables to be reused. The function should return the generated 28 x 28 x out_channel_dim images.

def generator(z, out_channel_dim, is_train=True):
    """
    Create the generator network
    :param z: Input z
    :param out_channel_dim: The number of channels in the output image
    :param is_train: Boolean if generator is being used for training
    :return: The tensor output of the generator
    """
    reuse = not is_train
    with tf.variable_scope('generator', reuse=reuse):
        # First layer is fully connected.
        layer_size = 7*7*512
        x = tf.layers.dense(inputs=z, units=layer_size)

        # Reshape into a convolutional layer.
        c1 = tf.reshape(x, [-1, 7, 7, 512])
        c1 = tf.layers.batch_normalization(c1, training=is_train)
        c1 = tf.maximum(0.2*c1, c1)
        
        # Input is 7x7x512
        c2 = tf.layers.conv2d_transpose(c1, 256, 5, [2,2], activation=None, padding='same')
        c2 = tf.layers.batch_normalization(c2, training=is_train)
        c2 = tf.maximum(0.2*c2, c2)
        
        # Input is 14x14x256
        c3 = tf.layers.conv2d_transpose(c2, 128, 5, [2,2], activation=None, padding='same')
        c3 = tf.layers.batch_normalization(c3, training=is_train)
        c3 = tf.maximum(0.2*c3, c3)
        
        # Convert to 28x28x256 to 28x28xout_channel_dim
        logits = tf.layers.conv2d_transpose(c3, out_channel_dim, 5, [1,1], activation=None, padding='same')
        
        # Discriminator excepts -1 to 1 -> apply tanh -> -0.5 to 0.5 since images are scaled to this range.
        output = 0.5*tf.tanh(logits)
    
        return output


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_generator(generator, tf)

Tests Passed

Loss¶

Implement model_loss to build the GANs for training and calculate the loss. The function should return a tuple of (discriminator loss, generator loss). Use the following functions you implemented:

discriminator(images, reuse=False)
generator(z, out_channel_dim, is_train=True)

def model_loss(input_real, input_z, out_channel_dim):
    """
    Get the loss for the discriminator and generator
    :param input_real: Images from the real dataset
    :param input_z: Z input
    :param out_channel_dim: The number of channels in the output image
    :return: A tuple of (discriminator loss, generator loss)
    """
    
    g_model = generator(input_z, out_channel_dim)
    d_model_real, d_logits_real = discriminator(input_real)
    d_model_fake, d_logits_fake = discriminator(g_model, True)
    
    d_real_loss = tf.reduce_mean(
        tf.nn.sigmoid_cross_entropy_with_logits(logits=d_logits_real, labels=tf.ones_like(d_model_real)))
    d_fake_loss = tf.reduce_mean(
        tf.nn.sigmoid_cross_entropy_with_logits(logits=d_logits_fake, labels=tf.zeros_like(d_model_fake)))
    g_loss = tf.reduce_mean(
        tf.nn.sigmoid_cross_entropy_with_logits(logits=d_logits_fake, labels=tf.ones_like(d_model_fake)))
    
    d_loss = d_real_loss + d_fake_loss
    
    return d_loss, g_loss


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_loss(model_loss)

Tests Passed

Optimization¶

Implement model_opt to create the optimization operations for the GANs. Use tf.trainable_variables to get all the trainable variables. Filter the variables with names that are in the discriminator and generator scope names. The function should return a tuple of (discriminator training operation, generator training operation).

def model_opt(d_loss, g_loss, learning_rate, beta1):
    """
    Get optimization operations
    :param d_loss: Discriminator loss Tensor
    :param g_loss: Generator loss Tensor
    :param learning_rate: Learning Rate Placeholder
    :param beta1: The exponential decay rate for the 1st moment in the optimizer
    :return: A tuple of (discriminator training operation, generator training operation)
    """

    # Get weights and bias to update.
    t_vars = tf.trainable_variables()
    d_vars = [var for var in t_vars if var.name.startswith('discriminator')]
    g_vars = [var for var in t_vars if var.name.startswith('generator')]
    
    # Optimize
    with tf.control_dependencies(tf.get_collection(tf.GraphKeys.UPDATE_OPS)):
        d_train_opt = tf.train.AdamOptimizer(learning_rate=learning_rate, beta1=beta1).minimize(d_loss, var_list=d_vars)
        g_train_opt = tf.train.AdamOptimizer(learning_rate=learning_rate, beta1=beta1).minimize(g_loss, var_list=g_vars)
        
    return d_train_opt, g_train_opt

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_opt(model_opt, tf)

Tests Passed

Neural Network Training¶

Show Output¶

Use this function to show the current output of the generator during training. It will help you determine how well the GANs is training.

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import numpy as np

def show_generator_output(sess, n_images, input_z, out_channel_dim, image_mode):
    """
    Show example output for the generator
    :param sess: TensorFlow session
    :param n_images: Number of Images to display
    :param input_z: Input Z Tensor
    :param out_channel_dim: The number of channels in the output image
    :param image_mode: The mode to use for images ("RGB" or "L")
    """
    cmap = None if image_mode == 'RGB' else 'gray'
    z_dim = input_z.get_shape().as_list()[-1]
    example_z = np.random.uniform(-1, 1, size=[n_images, z_dim])

    samples = sess.run(
        generator(input_z, out_channel_dim, False),
        feed_dict={input_z: example_z})

    images_grid = helper.images_square_grid(samples, image_mode)
    pyplot.imshow(images_grid, cmap=cmap)
    pyplot.show()

Train¶

Implement train to build and train the GANs. Use the following functions you implemented:

model_inputs(image_width, image_height, image_channels, z_dim)
model_loss(input_real, input_z, out_channel_dim)
model_opt(d_loss, g_loss, learning_rate, beta1)

Use the show_generator_output to show generator output while you train. Running show_generator_output for every batch will drastically increase training time and increase the size of the notebook. It's recommended to print the generator output every 100 batches.

def train(epoch_count, batch_size, z_dim, learning_rate, beta1, get_batches, data_shape, data_image_mode, freq_see_loss = 10):
    """
    Train the GAN
    :param epoch_count: Number of epochs
    :param batch_size: Batch Size
    :param z_dim: Z dimension
    :param learning_rate: Learning Rate
    :param beta1: The exponential decay rate for the 1st moment in the optimizer
    :param get_batches: Function to get batches
    :param data_shape: Shape of the data
    :param data_image_mode: The image mode to use for images ("RGB" or "L")
    """
    
    # Get the out_channel_dim.
    if data_image_mode == "RGB":
        out_channel_dim = 3
    elif data_image_mode == "L":
        out_channel_dim = 1
        
    # Get the image dimensions.
    image_width = data_shape[-2]
    image_height = data_shape[-3]
        
    # Get the model inputs.
    real_input, z_data, learning_rate_placeholder = model_inputs(image_width, image_height, out_channel_dim, z_dim)
    
    # Obtain the losses.
    d_loss, g_loss = model_loss(real_input, z_data, out_channel_dim)
    
    # Obtain the model optimization operations.
    d_train_opt, g_train_opt = model_opt(d_loss, g_loss, learning_rate, beta1)
    
    # Container to hold the losses.
    g_losses = []
    d_losses = []
    steps = 0
    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        for epoch_i in range(epoch_count):
            for batch_images in get_batches(batch_size):
                steps+=1
                
                # Generate the random z vector.
#                 z_data_input = tf.random_uniform([batch_size, z_dim], -1, 1)
                z_data_input = np.random.uniform(-1, 1, size=(batch_size, z_dim))
                
                sess.run(d_train_opt, feed_dict={real_input:batch_images, z_data:z_data_input})
                sess.run(g_train_opt, feed_dict={real_input:batch_images, z_data:z_data_input})
                
                if steps % freq_see_loss == 0:
                    d_loss_out, g_loss_out = sess.run([d_loss, g_loss], feed_dict={real_input:batch_images, z_data:z_data_input})
                    
                    print('Epoch: {} -- Step: {} -- d_loss: {} -- g_loss: {}'.format(epoch_i, steps, d_loss_out, g_loss_out))
                    g_losses.append(g_loss_out)
                    d_losses.append(d_loss_out)
                
                if steps % 100 == 0:
                    z_data_input_tensor = tf.random_uniform([16, z_dim], -1, 1)
                    show_generator_output(sess, 16, z_data_input_tensor, out_channel_dim, data_image_mode)
                    
    pyplot.figure()
    pyplot.plot(g_losses)
    pyplot.title('g_losses')
    pyplot.figure()
    pyplot.plot(d_losses)
    pyplot.title('d_losses')

MNIST¶

Test your GANs architecture on MNIST. After 2 epochs, the GANs should be able to generate images that look like handwritten digits. Make sure the loss of the generator is lower than the loss of the discriminator or close to 0.

batch_size = 128
z_dim = 100
learning_rate = 0.0002
beta1 = 0.5


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
epochs = 2

mnist_dataset = helper.Dataset('mnist', glob(os.path.join(data_dir, 'mnist/*.jpg')))

with tf.Graph().as_default():
    train(epochs, batch_size, z_dim, learning_rate, beta1, mnist_dataset.get_batches,
          mnist_dataset.shape, mnist_dataset.image_mode)

Epoch: 0 -- Step: 10 -- d_loss: 0.2919052541255951 -- g_loss: 4.940899848937988
Epoch: 0 -- Step: 20 -- d_loss: 0.022066690027713776 -- g_loss: 4.52243185043335
Epoch: 0 -- Step: 30 -- d_loss: 0.02281767688691616 -- g_loss: 6.639409065246582
Epoch: 0 -- Step: 40 -- d_loss: 0.030390817672014236 -- g_loss: 8.16735553741455
Epoch: 0 -- Step: 50 -- d_loss: 4.331295967102051 -- g_loss: 10.53857135772705
Epoch: 0 -- Step: 60 -- d_loss: 0.7680845260620117 -- g_loss: 1.0419775247573853
Epoch: 0 -- Step: 70 -- d_loss: 0.2806212604045868 -- g_loss: 2.7464795112609863
Epoch: 0 -- Step: 80 -- d_loss: 0.594424843788147 -- g_loss: 1.356933832168579
Epoch: 0 -- Step: 90 -- d_loss: 0.7611536979675293 -- g_loss: 1.7591285705566406
Epoch: 0 -- Step: 100 -- d_loss: 1.1949225664138794 -- g_loss: 0.6835594177246094

Epoch: 0 -- Step: 110 -- d_loss: 1.532126784324646 -- g_loss: 0.47429269552230835
Epoch: 0 -- Step: 120 -- d_loss: 0.9302685260772705 -- g_loss: 1.0733888149261475
Epoch: 0 -- Step: 130 -- d_loss: 1.155641794204712 -- g_loss: 0.9005465507507324
Epoch: 0 -- Step: 140 -- d_loss: 1.834296703338623 -- g_loss: 0.2523607909679413
Epoch: 0 -- Step: 150 -- d_loss: 1.1264755725860596 -- g_loss: 0.6817317008972168
Epoch: 0 -- Step: 160 -- d_loss: 1.051430344581604 -- g_loss: 0.8035173416137695
Epoch: 0 -- Step: 170 -- d_loss: 1.0306580066680908 -- g_loss: 0.7705690860748291
Epoch: 0 -- Step: 180 -- d_loss: 1.166183590888977 -- g_loss: 1.5168412923812866
Epoch: 0 -- Step: 190 -- d_loss: 1.1633694171905518 -- g_loss: 0.9853212833404541
Epoch: 0 -- Step: 200 -- d_loss: 1.1337440013885498 -- g_loss: 0.6978400945663452

Epoch: 0 -- Step: 210 -- d_loss: 1.0198578834533691 -- g_loss: 1.161963701248169
Epoch: 0 -- Step: 220 -- d_loss: 1.0460155010223389 -- g_loss: 0.9255943894386292
Epoch: 0 -- Step: 230 -- d_loss: 1.0392814874649048 -- g_loss: 1.3703691959381104
Epoch: 0 -- Step: 240 -- d_loss: 1.197930932044983 -- g_loss: 0.5464293956756592
Epoch: 0 -- Step: 250 -- d_loss: 0.9769629240036011 -- g_loss: 1.4929358959197998
Epoch: 0 -- Step: 260 -- d_loss: 0.9208825826644897 -- g_loss: 1.2941476106643677
Epoch: 0 -- Step: 270 -- d_loss: 0.9873843193054199 -- g_loss: 1.5691750049591064
Epoch: 0 -- Step: 280 -- d_loss: 0.9561924934387207 -- g_loss: 1.158470869064331
Epoch: 0 -- Step: 290 -- d_loss: 1.0848631858825684 -- g_loss: 1.8248838186264038
Epoch: 0 -- Step: 300 -- d_loss: 0.9550783634185791 -- g_loss: 1.5365937948226929

Epoch: 0 -- Step: 310 -- d_loss: 0.9725456833839417 -- g_loss: 0.7777073383331299
Epoch: 0 -- Step: 320 -- d_loss: 1.6753404140472412 -- g_loss: 2.4892797470092773
Epoch: 0 -- Step: 330 -- d_loss: 1.0453178882598877 -- g_loss: 1.0394611358642578
Epoch: 0 -- Step: 340 -- d_loss: 1.359913945198059 -- g_loss: 0.39264750480651855
Epoch: 0 -- Step: 350 -- d_loss: 0.9763960242271423 -- g_loss: 0.9105708599090576
Epoch: 0 -- Step: 360 -- d_loss: 1.1472727060317993 -- g_loss: 0.5380191802978516
Epoch: 0 -- Step: 370 -- d_loss: 0.983116090297699 -- g_loss: 1.0143802165985107
Epoch: 0 -- Step: 380 -- d_loss: 1.1879642009735107 -- g_loss: 0.5547511577606201
Epoch: 0 -- Step: 390 -- d_loss: 0.9911061525344849 -- g_loss: 1.1106277704238892
Epoch: 0 -- Step: 400 -- d_loss: 1.033574104309082 -- g_loss: 1.387387990951538

Epoch: 0 -- Step: 410 -- d_loss: 0.9704635143280029 -- g_loss: 0.8986825942993164
Epoch: 0 -- Step: 420 -- d_loss: 1.0224109888076782 -- g_loss: 1.1413966417312622
Epoch: 0 -- Step: 430 -- d_loss: 0.94695645570755 -- g_loss: 1.0982029438018799
Epoch: 0 -- Step: 440 -- d_loss: 0.890794575214386 -- g_loss: 1.3059253692626953
Epoch: 0 -- Step: 450 -- d_loss: 1.0167725086212158 -- g_loss: 0.8158342838287354
Epoch: 0 -- Step: 460 -- d_loss: 0.9179112911224365 -- g_loss: 1.2322192192077637
Epoch: 1 -- Step: 470 -- d_loss: 1.1071112155914307 -- g_loss: 0.5699755549430847
Epoch: 1 -- Step: 480 -- d_loss: 1.009651780128479 -- g_loss: 1.1748476028442383
Epoch: 1 -- Step: 490 -- d_loss: 0.98896324634552 -- g_loss: 0.7256466150283813
Epoch: 1 -- Step: 500 -- d_loss: 1.0049916505813599 -- g_loss: 0.7055234909057617

Epoch: 1 -- Step: 510 -- d_loss: 0.9607968926429749 -- g_loss: 1.5129117965698242
Epoch: 1 -- Step: 520 -- d_loss: 0.9008792638778687 -- g_loss: 1.2569299936294556
Epoch: 1 -- Step: 530 -- d_loss: 1.0684170722961426 -- g_loss: 1.382836103439331
Epoch: 1 -- Step: 540 -- d_loss: 0.9576581716537476 -- g_loss: 1.3073469400405884
Epoch: 1 -- Step: 550 -- d_loss: 0.9453575611114502 -- g_loss: 1.049583077430725
Epoch: 1 -- Step: 560 -- d_loss: 1.0407055616378784 -- g_loss: 0.7289315462112427
Epoch: 1 -- Step: 570 -- d_loss: 0.9388839602470398 -- g_loss: 1.0104011297225952
Epoch: 1 -- Step: 580 -- d_loss: 1.0753720998764038 -- g_loss: 1.5687161684036255
Epoch: 1 -- Step: 590 -- d_loss: 0.9824951887130737 -- g_loss: 1.000942349433899
Epoch: 1 -- Step: 600 -- d_loss: 1.0155689716339111 -- g_loss: 0.8422719836235046

Epoch: 1 -- Step: 610 -- d_loss: 1.1190448999404907 -- g_loss: 1.241908311843872
Epoch: 1 -- Step: 620 -- d_loss: 1.0187878608703613 -- g_loss: 1.0736923217773438
Epoch: 1 -- Step: 630 -- d_loss: 1.017331600189209 -- g_loss: 0.7904312610626221
Epoch: 1 -- Step: 640 -- d_loss: 1.2177494764328003 -- g_loss: 0.5454082489013672
Epoch: 1 -- Step: 650 -- d_loss: 1.0041112899780273 -- g_loss: 0.7869515419006348
Epoch: 1 -- Step: 660 -- d_loss: 1.0074472427368164 -- g_loss: 0.7443562150001526
Epoch: 1 -- Step: 670 -- d_loss: 0.9385982751846313 -- g_loss: 1.08158278465271
Epoch: 1 -- Step: 680 -- d_loss: 0.9918556809425354 -- g_loss: 1.1653330326080322
Epoch: 1 -- Step: 690 -- d_loss: 0.9793219566345215 -- g_loss: 0.7843822836875916
Epoch: 1 -- Step: 700 -- d_loss: 1.4795786142349243 -- g_loss: 0.34337317943573

Epoch: 1 -- Step: 710 -- d_loss: 0.93310546875 -- g_loss: 1.6393282413482666
Epoch: 1 -- Step: 720 -- d_loss: 0.8216481804847717 -- g_loss: 1.1438655853271484
Epoch: 1 -- Step: 730 -- d_loss: 0.8867435455322266 -- g_loss: 1.4032829999923706
Epoch: 1 -- Step: 740 -- d_loss: 0.8842298984527588 -- g_loss: 1.3297260999679565
Epoch: 1 -- Step: 750 -- d_loss: 0.9585742354393005 -- g_loss: 0.7389783263206482
Epoch: 1 -- Step: 760 -- d_loss: 0.8262853622436523 -- g_loss: 1.0515427589416504
Epoch: 1 -- Step: 770 -- d_loss: 1.0015227794647217 -- g_loss: 1.4153945446014404
Epoch: 1 -- Step: 780 -- d_loss: 0.9721543788909912 -- g_loss: 0.8989022970199585
Epoch: 1 -- Step: 790 -- d_loss: 0.875064492225647 -- g_loss: 0.9502137899398804
Epoch: 1 -- Step: 800 -- d_loss: 0.8562411665916443 -- g_loss: 0.9120610356330872

Epoch: 1 -- Step: 810 -- d_loss: 0.9958223700523376 -- g_loss: 0.6282020211219788
Epoch: 1 -- Step: 820 -- d_loss: 0.9233318567276001 -- g_loss: 0.8647503852844238
Epoch: 1 -- Step: 830 -- d_loss: 1.004600167274475 -- g_loss: 2.060114622116089
Epoch: 1 -- Step: 840 -- d_loss: 0.9357266426086426 -- g_loss: 0.8173389434814453
Epoch: 1 -- Step: 850 -- d_loss: 0.8042709231376648 -- g_loss: 1.0028992891311646
Epoch: 1 -- Step: 860 -- d_loss: 0.9073479771614075 -- g_loss: 0.9639356136322021
Epoch: 1 -- Step: 870 -- d_loss: 0.8810655474662781 -- g_loss: 0.8622294664382935
Epoch: 1 -- Step: 880 -- d_loss: 0.8318590521812439 -- g_loss: 1.4579449892044067
Epoch: 1 -- Step: 890 -- d_loss: 0.9475924968719482 -- g_loss: 0.7071715593338013
Epoch: 1 -- Step: 900 -- d_loss: 1.0195789337158203 -- g_loss: 0.6908923983573914

Epoch: 1 -- Step: 910 -- d_loss: 0.8795006275177002 -- g_loss: 0.8145804405212402
Epoch: 1 -- Step: 920 -- d_loss: 1.038091778755188 -- g_loss: 0.6284948587417603
Epoch: 1 -- Step: 930 -- d_loss: 0.7910155653953552 -- g_loss: 1.2550705671310425

CelebA¶

Run your GANs on CelebA. It will take around 20 minutes on the average GPU to run one epoch. You can run the whole epoch or stop when it starts to generate realistic faces.

batch_size = 128
z_dim = 100
learning_rate = 0.0002
beta1 = 0.5


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
epochs = 20

celeba_dataset = helper.Dataset('celeba', glob(os.path.join(data_dir, 'img_align_celeba/*.jpg')))
with tf.Graph().as_default():
    train(epochs, batch_size, z_dim, learning_rate, beta1, celeba_dataset.get_batches,
          celeba_dataset.shape, celeba_dataset.image_mode)

Epoch: 0 -- Step: 10 -- d_loss: 0.6189572811126709 -- g_loss: 1.6682405471801758
Epoch: 0 -- Step: 20 -- d_loss: 0.8532211780548096 -- g_loss: 2.508634567260742
Epoch: 0 -- Step: 30 -- d_loss: 0.9677637219429016 -- g_loss: 2.884580135345459
Epoch: 0 -- Step: 40 -- d_loss: 1.280186414718628 -- g_loss: 0.5198928117752075
Epoch: 0 -- Step: 50 -- d_loss: 1.1457328796386719 -- g_loss: 0.5005475878715515
Epoch: 0 -- Step: 60 -- d_loss: 0.6648433804512024 -- g_loss: 2.4754750728607178
Epoch: 0 -- Step: 70 -- d_loss: 0.4169224202632904 -- g_loss: 1.8632996082305908
Epoch: 0 -- Step: 80 -- d_loss: 0.9443435072898865 -- g_loss: 0.7916808724403381
Epoch: 0 -- Step: 90 -- d_loss: 0.47528398036956787 -- g_loss: 1.4359049797058105
Epoch: 0 -- Step: 100 -- d_loss: 1.1192724704742432 -- g_loss: 0.7107819318771362

Epoch: 0 -- Step: 110 -- d_loss: 0.9428191781044006 -- g_loss: 0.7804093360900879
Epoch: 0 -- Step: 120 -- d_loss: 0.28318312764167786 -- g_loss: 2.026604175567627
Epoch: 0 -- Step: 130 -- d_loss: 0.3575264513492584 -- g_loss: 2.76959228515625
Epoch: 0 -- Step: 140 -- d_loss: 0.8777046203613281 -- g_loss: 0.7794727087020874
Epoch: 0 -- Step: 150 -- d_loss: 0.22578366100788116 -- g_loss: 3.62062406539917
Epoch: 0 -- Step: 160 -- d_loss: 0.13888703286647797 -- g_loss: 4.594459056854248
Epoch: 0 -- Step: 170 -- d_loss: 0.3556753396987915 -- g_loss: 1.8537588119506836
Epoch: 0 -- Step: 180 -- d_loss: 0.21600377559661865 -- g_loss: 2.78483247756958
Epoch: 0 -- Step: 190 -- d_loss: 0.7178511619567871 -- g_loss: 1.0621424913406372
Epoch: 0 -- Step: 200 -- d_loss: 1.7409213781356812 -- g_loss: 0.3326241374015808

Epoch: 0 -- Step: 210 -- d_loss: 1.1162914037704468 -- g_loss: 0.6142660975456238
Epoch: 0 -- Step: 220 -- d_loss: 1.187780737876892 -- g_loss: 0.5684333443641663
Epoch: 0 -- Step: 230 -- d_loss: 0.31298547983169556 -- g_loss: 2.8665857315063477
Epoch: 0 -- Step: 240 -- d_loss: 0.3061736226081848 -- g_loss: 4.465298652648926
Epoch: 0 -- Step: 250 -- d_loss: 0.5903671979904175 -- g_loss: 1.8480724096298218
Epoch: 0 -- Step: 260 -- d_loss: 0.46069276332855225 -- g_loss: 2.3626768589019775
Epoch: 0 -- Step: 270 -- d_loss: 1.2914705276489258 -- g_loss: 1.207122802734375
Epoch: 0 -- Step: 280 -- d_loss: 0.40697237849235535 -- g_loss: 2.079935312271118
Epoch: 0 -- Step: 290 -- d_loss: 0.7015174627304077 -- g_loss: 0.9863153696060181
Epoch: 0 -- Step: 300 -- d_loss: 3.0196330547332764 -- g_loss: 4.714893341064453

Epoch: 0 -- Step: 310 -- d_loss: 0.684442937374115 -- g_loss: 1.6667400598526
Epoch: 0 -- Step: 320 -- d_loss: 1.2834985256195068 -- g_loss: 0.3918673098087311
Epoch: 0 -- Step: 330 -- d_loss: 1.095014214515686 -- g_loss: 0.5890215635299683
Epoch: 0 -- Step: 340 -- d_loss: 1.3010108470916748 -- g_loss: 3.9024429321289062
Epoch: 0 -- Step: 350 -- d_loss: 1.0750625133514404 -- g_loss: 1.6504697799682617
Epoch: 0 -- Step: 360 -- d_loss: 0.8633740544319153 -- g_loss: 1.1450884342193604
Epoch: 0 -- Step: 370 -- d_loss: 0.655682384967804 -- g_loss: 1.1551790237426758
Epoch: 0 -- Step: 380 -- d_loss: 0.32378149032592773 -- g_loss: 1.840850830078125
Epoch: 0 -- Step: 390 -- d_loss: 0.5401840209960938 -- g_loss: 2.5258617401123047
Epoch: 0 -- Step: 400 -- d_loss: 1.4231135845184326 -- g_loss: 1.0094537734985352

Epoch: 0 -- Step: 410 -- d_loss: 1.5186538696289062 -- g_loss: 0.3039957284927368
Epoch: 0 -- Step: 420 -- d_loss: 0.8933823704719543 -- g_loss: 1.017946481704712
Epoch: 0 -- Step: 430 -- d_loss: 0.5030015707015991 -- g_loss: 1.7307782173156738
Epoch: 0 -- Step: 440 -- d_loss: 0.6974966526031494 -- g_loss: 1.0915714502334595
Epoch: 0 -- Step: 450 -- d_loss: 1.3992705345153809 -- g_loss: 2.4568943977355957
Epoch: 0 -- Step: 460 -- d_loss: 1.432362675666809 -- g_loss: 0.3547859191894531
Epoch: 0 -- Step: 470 -- d_loss: 0.3885296881198883 -- g_loss: 3.3084805011749268
Epoch: 0 -- Step: 480 -- d_loss: 1.4424132108688354 -- g_loss: 3.7159488201141357
Epoch: 0 -- Step: 490 -- d_loss: 0.6583214998245239 -- g_loss: 1.8128461837768555
Epoch: 0 -- Step: 500 -- d_loss: 1.0541586875915527 -- g_loss: 1.4473506212234497

Epoch: 0 -- Step: 510 -- d_loss: 0.7510514259338379 -- g_loss: 2.025989294052124
Epoch: 0 -- Step: 520 -- d_loss: 0.4676828980445862 -- g_loss: 1.976776123046875
Epoch: 0 -- Step: 530 -- d_loss: 0.7727198600769043 -- g_loss: 1.4416558742523193
Epoch: 0 -- Step: 540 -- d_loss: 1.3622641563415527 -- g_loss: 1.1495091915130615
Epoch: 0 -- Step: 550 -- d_loss: 0.8533591032028198 -- g_loss: 1.0452699661254883
Epoch: 0 -- Step: 560 -- d_loss: 0.4185512661933899 -- g_loss: 2.6592907905578613
Epoch: 0 -- Step: 570 -- d_loss: 0.41667357087135315 -- g_loss: 1.8484811782836914
Epoch: 0 -- Step: 580 -- d_loss: 1.1975208520889282 -- g_loss: 1.084951400756836
Epoch: 0 -- Step: 590 -- d_loss: 0.40823841094970703 -- g_loss: 1.647853970527649
Epoch: 0 -- Step: 600 -- d_loss: 1.4130827188491821 -- g_loss: 0.6910707354545593

Epoch: 0 -- Step: 610 -- d_loss: 0.39678677916526794 -- g_loss: 2.560805320739746
Epoch: 0 -- Step: 620 -- d_loss: 0.6609561443328857 -- g_loss: 1.0886285305023193
Epoch: 0 -- Step: 630 -- d_loss: 1.5914608240127563 -- g_loss: 3.239980936050415
Epoch: 0 -- Step: 640 -- d_loss: 0.9681727886199951 -- g_loss: 1.6076397895812988
Epoch: 0 -- Step: 650 -- d_loss: 0.7274972200393677 -- g_loss: 2.520704507827759
Epoch: 0 -- Step: 660 -- d_loss: 0.8910841941833496 -- g_loss: 0.8940232992172241
Epoch: 0 -- Step: 670 -- d_loss: 0.4553184509277344 -- g_loss: 1.965212106704712
Epoch: 0 -- Step: 680 -- d_loss: 0.6510272026062012 -- g_loss: 1.2685779333114624
Epoch: 0 -- Step: 690 -- d_loss: 0.8435059785842896 -- g_loss: 2.0915842056274414
Epoch: 0 -- Step: 700 -- d_loss: 0.8578300476074219 -- g_loss: 0.823190450668335

Epoch: 0 -- Step: 710 -- d_loss: 0.4227743446826935 -- g_loss: 2.2841506004333496
Epoch: 0 -- Step: 720 -- d_loss: 0.39046141505241394 -- g_loss: 3.2438244819641113
Epoch: 0 -- Step: 730 -- d_loss: 1.5941988229751587 -- g_loss: 4.497476577758789
Epoch: 0 -- Step: 740 -- d_loss: 0.7174882888793945 -- g_loss: 1.9809353351593018
Epoch: 0 -- Step: 750 -- d_loss: 0.40130746364593506 -- g_loss: 2.518244743347168
Epoch: 0 -- Step: 760 -- d_loss: 0.633373498916626 -- g_loss: 3.046576976776123
Epoch: 0 -- Step: 770 -- d_loss: 0.9137560129165649 -- g_loss: 1.9444615840911865
Epoch: 0 -- Step: 780 -- d_loss: 0.6419381499290466 -- g_loss: 2.358516216278076
Epoch: 0 -- Step: 790 -- d_loss: 1.4742600917816162 -- g_loss: 0.594638466835022
Epoch: 0 -- Step: 800 -- d_loss: 0.9166625142097473 -- g_loss: 2.0175764560699463

Epoch: 0 -- Step: 810 -- d_loss: 0.7758190035820007 -- g_loss: 0.730849027633667
Epoch: 0 -- Step: 820 -- d_loss: 1.4578440189361572 -- g_loss: 0.3882875442504883
Epoch: 0 -- Step: 830 -- d_loss: 0.44379448890686035 -- g_loss: 2.494309186935425
Epoch: 0 -- Step: 840 -- d_loss: 0.3009606599807739 -- g_loss: 3.6628260612487793
Epoch: 0 -- Step: 850 -- d_loss: 0.5243213772773743 -- g_loss: 1.4126404523849487
Epoch: 0 -- Step: 860 -- d_loss: 1.6131051778793335 -- g_loss: 1.1347616910934448
Epoch: 0 -- Step: 870 -- d_loss: 1.640211582183838 -- g_loss: 0.30656397342681885
Epoch: 0 -- Step: 880 -- d_loss: 1.8730353116989136 -- g_loss: 0.24091392755508423
Epoch: 0 -- Step: 890 -- d_loss: 0.7271528840065002 -- g_loss: 1.2307286262512207
Epoch: 0 -- Step: 900 -- d_loss: 0.45269912481307983 -- g_loss: 2.506349563598633

Epoch: 0 -- Step: 910 -- d_loss: 1.7205414772033691 -- g_loss: 0.3101435899734497
Epoch: 0 -- Step: 920 -- d_loss: 0.778600811958313 -- g_loss: 1.340665340423584
Epoch: 0 -- Step: 930 -- d_loss: 0.416753888130188 -- g_loss: 3.8478164672851562
Epoch: 0 -- Step: 940 -- d_loss: 0.6839836835861206 -- g_loss: 1.671226143836975
Epoch: 0 -- Step: 950 -- d_loss: 0.21385543048381805 -- g_loss: 3.712894916534424
Epoch: 0 -- Step: 960 -- d_loss: 1.444953441619873 -- g_loss: 0.5507619380950928
Epoch: 0 -- Step: 970 -- d_loss: 1.3159905672073364 -- g_loss: 0.5107996463775635
Epoch: 0 -- Step: 980 -- d_loss: 1.3881958723068237 -- g_loss: 0.5218123197555542
Epoch: 0 -- Step: 990 -- d_loss: 1.3753917217254639 -- g_loss: 0.6706399917602539
Epoch: 0 -- Step: 1000 -- d_loss: 1.1528619527816772 -- g_loss: 0.8228816986083984

Submitting This Project¶

When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as "dlnd_face_generation.ipynb" and save it as a HTML file under "File" -> "Download as". Include the "helper.py" and "problem_unittests.py" files in your submission.